Recently, as various Internet techniques and multimedia have been developed rapidly, amounts of multimedia data also increase exponentially as well. Therefore, effective management and retrieval of multimedia data are necessitated.
However, since amounts of multimedia data are enormous and multimedia data are mixed with various types of image, video, audio, text and so forth, it is practically impossible to retrieve relevant multimedia directly from a multimedia database.
Therefore, it becomes necessary to develop a technology for retrieving and managing multimedia data effectively. Among those key technologies, an important one is multimedia index description that extracts index information to be used for retrieval and exploration.
In other words, a user should be able to retrieve a specific multimedia data through a pre-processing procedure by extracting a descriptor that describes a unique characteristics of each multimedia data when a multimedia database is constructed; and a procedure for computing a similarity distance between the descriptor of query multimedia data that the user requests and each descriptor of data in a multimedia database.
Because of indispensability for multimedia data retrieval, International Organization for Standardization (ISO), International Electrotechnical Commission (IEC) and Joint Technical Committee 1 (ISO/IEC JTC 1) set forth a standard for content-based multimedia retrieval technology with regard to Moving Picture Experts Group-7 (MPEG-7).
Currently, pieces of information on such characteristics as shape, color, text, motion and so on are used for multimedia data description.
Meanwhile, motion information is an important characteristics for retrieving video data. Video data retrieval is a method for retrieving similar video data by extracting a motion descriptor which describes characteristic motions of an object expressed by sequences that constitutes video data and estimating a similarity distance between query video data inputted by an user and the motion descriptor of the video data stored into a database.
In this case, there are various types of the motion descriptor; that are, a camera motion which describes various motions of a camera, a motion trajectory which describes a trajectory of a moving object, a parametric motion which describes motions of a whole image and a motion activity which expresses quantitatively activeness of an image motion. Effectiveness of the video data retrieval method with use of the motion descriptor depends on ability of the descriptor in how well it can describe characteristics of video data.
The motion trajectory descriptor can be one of the most frequently used descriptor to describe a spatiotemporal trajectory of a moving object. The motion trajectory descriptor is classified as a global motion and an object motion. The global motion represents a camera motion, i.e., motions of the camera and the object motion represents an object in a user's interest, i.e., motions of the object.
The global motion describes a motion with a centroid of a square that minimally encompasses a corresponding object. In this case, based on information about an object's position, velocity, acceleration and so on, a trajectory of an x direction of the centroid in the moving object is expressed with a value shown from the following equation 1.
                              ∀                      t            ∈                          [                                                t                  0                                ,                                  t                  1                                            ]                                      ,                              x            ⁡                          (                              t                -                                  t                  0                                            )                                =                                    x              0                        +                                          v                x                            ⁡                              (                                  t                  -                                      t                    0                                                  )                                      +                                          1                2                            ⁢                                                                    a                    x                                    ⁡                                      (                                          t                      -                                              t                        0                                                              )                                                  2                                                                        Eq        .                                  ⁢                  (          1          )                    
x0: a position when t=t0 
x(t−t0): x cordinate
νx: velocity
αx: acceleration
Similar to the X direction, y and z directions of the centroid are expressed as the following equation 2.
                                                                        ∀                                  t                  ∈                                      [                                                                  t                        0                                            ,                                              t                        1                                                              ]                                                              ,                                                y                  ⁡                                      (                                          t                      -                                              t                        0                                                              )                                                  =                                                      y                    0                                    +                                                            V                      y                                        ⁡                                          (                                              t                        -                                                  t                          0                                                                    )                                                        +                                                            1                      2                                        ⁢                                                                                            a                          y                                                ⁡                                                  (                                                      t                            -                                                          t                              0                                                                                )                                                                    2                                                                                                                                                              z                ⁡                                  (                                      t                    -                                          t                      0                                                        )                                            =                                                z                  0                                +                                                      v                    z                                    ⁡                                      (                                          t                      -                                              t                        0                                                              )                                                  +                                                      1                    2                                    ⁢                                                                                    a                        z                                            ⁡                                              (                                                  t                          -                                                      t                            0                                                                          )                                                              2                                                                                                          Eq        .                                  ⁢                  (          2          )                    
y(t−t0): y cordinate
z(t−t0): zoom-in/out of a camera
That is, the global motion represents a characteristic that expresses a level of velocity with which the object is moving at certain two points.
A distance between two general motion trajectories of the objects is expressed as a following equation 3.
                              d          ⁡                      (                          D1              ,              D2                        )                          =                              ∑            i                    ⁢                      [                                                                                                                                                                                    α                            ⁢                                                                                                                                                                (                                                                                                                  x1                                        i                                                                            -                                                                              x2                                        i                                                                                                              )                                                                    2                                                                +                                                                                                      (                                                                                                                  y1                                        i                                                                            -                                                                              y2                                        i                                                                                                              )                                                                    2                                                                +                                                                                                      (                                                                                                                  z1                                        i                                                                            -                                                                              z2                                        i                                                                                                              )                                                                    2                                                                                                                            Δ                                ⁢                                                                                                                                  ⁢                                                                  t                                  i                                                                                                                                              +                                                                                                                                                                                          β                            ⁢                                                                                                                                                                (                                                                                                                  v1                                                                                  x                                          ⁢                                                                                                                                                                          ⁢                                          i                                                                                                                    -                                                                              v2                                                                                  x                                          ⁢                                                                                                                                                                          ⁢                                          i                                                                                                                                                      )                                                                    2                                                                +                                                                                                      (                                                                                                                  v1                                                                                  y                                          ⁢                                                                                                                                                                          ⁢                                          i                                                                                                                    -                                                                              v2                                                                                  y                                          ⁢                                                                                                                                                                          ⁢                                          i                                                                                                                                                      )                                                                    2                                                                +                                                                                                      (                                                                                                                  v1                                                                                  z                                          ⁢                                                                                                                                                                          ⁢                                          i                                                                                                                    -                                                                              v2                                                                                  z                                          ⁢                                                                                                                                                                          ⁢                                          i                                                                                                                                                      )                                                                    2                                                                                                                            Δ                                ⁢                                                                                                                                  ⁢                                                                  t                                  i                                                                                                                                              +                                                                                                                                                                                      χ                    ⁢                                                                                                                        (                                                                                          a1                                                                  x                                  ⁢                                                                                                                                          ⁢                                  i                                                                                            -                                                              a2                                                                  x                                  ⁢                                                                                                                                          ⁢                                  i                                                                                                                      )                                                    2                                                +                                                                              (                                                                                          a1                                                                  y                                  ⁢                                                                                                                                          ⁢                                  i                                                                                            -                                                              a2                                                                  y                                  ⁢                                                                                                                                          ⁢                                  i                                                                                                                      )                                                    2                                                +                                                                              (                                                                                          a1                                                                  z                                  ⁢                                                                                                                                          ⁢                                  i                                                                                            -                                                              a2                                                                  z                                  ⁢                                                                                                                                          ⁢                                  i                                                                                                                      )                                                    2                                                                                            Δ                        ⁢                                                                                                  ⁢                                                  t                          i                                                                                                                                          ]                                              Eq        .                                  ⁢                  (          3          )                    
Δti: time at i th characteristic
α,β,χ: weight
However, a method for content-based video data retrieval through the use of the conventional motion trajectory descriptor has a characteristic merely about the global motion and this means that the above descriptor describes solely a motion trajectory of an object without any information on motions of the object. Because of this characteristic, identical motion trajectories of the objects that possess different shapes and motions express the same characteristic, resulting in the following limitations.
Firstly, a characteristic of a user's perception cannot be accurately reflected and motions of the object, i.e., shapes of the object that change in accordance with the time cannot be described. As a result, in case that characteristics with respect to the global motion are similar, the video data, that the user feels different because they are originated from different objects, are retrieved incorrectly as similar video data. In other words, the motion trajectory descriptor describes only motions of the object without information on shapes of the object, and thus, the same characteristic is expressed even for the object having the same motion but the different shape. For instance, in a perspective of human understanding, walking of a man and that of an animal are perceived as different motions, however, the motion trajectory descriptor represents them as the same characteristic motion.
Secondly, since motions of the object are different even for the identical object, the motion trajectory descriptor cannot discriminate motions of different objects in case that characteristics with respect to the global motion are similar although each has different image sequence data.